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Description 

Convertible runtime graphical user 

interface 

Cross Reference To Related Applications 

[000 1 ] 5,600,780/5,603,034/5,652,884/5,786,815/6,208,336 
Background of Invention 

[0002] The present invention field of endeavor is user interface 
management systems (UIMS). Graphical User Interface 
(GUI) is "the Front End", "the Face" of any application. 
Modern, sometimes called "configurable", GUIs allow the 
end-user to modify some of the GUI widget's attributes: 
geometrical sizes, position, background color, and text 
font. Unfortunately, they provide no possibility to change 
the behavior of the widget associated with some program 
variable at runtime, or create new link(s) between this 
variable and new widget(s). For example, it is impossible 
to convert the widget "editable integer" to the widget 
"slider" without changing the application source code 



and/or without compiling a new application executable. A 
UIMS that would allow the runtime creation of new widget 
shapes and/or conversion of a widget or a group of wid- 
gets to another widget or group of widgets without re- 
compilation and/or restart of the application executable 

would be desirable. 
Brief Description of Drawings 

[0003] FIG. 1 shows contents of the example program Hel- 
loWorld.cpp; 

[0004] FIG. 2 shows window with parameter "Param" when pro- 
gram HelloWorld.cpp runs for the first time; 

[0005] FIG. 3 shows window with new (converted) representation 
of parameter "Param"; 

[0006] FIG. 4 shows contents of the example program SumOfT- 
woNumbers.cpp; 

[0007] FIG. 5 shows window with parameters of the program 
from FIG. 4 when it runs for the first time; 

[0008] FIG. 6 shows window from FIG. 5 after the values of pa- 
rameters have been changed; 

[0009] FIG. 7 shows window with new (converted) representation 
of variables of the program presented in FIG. 4; 

[0010] FIG. 8 shows window presented by FIG. 7 with duplicated 
parameters; 



[0011] FIG. 9 shows window presented by FIG. 8 with parameters 

converted to sliders; 
[0012] FIG. 10 provides a list of members of the class CUIPara- 

m Entry; 

[0013] FIG. 11 provides a list of members of the class CUIObject; 

[0014] FIG. 12 provides a list of CUI Viewer commands currently 

supported in "Edit" mode; 
Detailed Description 

[0015] The current embodiment of this invention is a set of 

methods, procedures and functions implemented either in 
specialized Dynamic Link Library (CUI DLL) or standalone 
executable (CUI Viewer), and data representing Convert- 
ible User Interface Description (CUI Description). 

[0016] All variables of any application can be divided into two 
groups with respect to their relationship with the GUI. 
Some are associated with some widgets, and thus they are 
"visible" or perhaps even "changeable" in the GUI, and the 
others are not. We refer to GUI-"visible" variables as CUI 
variables and we treat them in a special way. First of all, 
we implemented a special procedure for their registration, 
which uses their names and types. This procedure uses a 
special memory manager (CUI Memory Manager) for their 



memory allocation. Second, we provide a set of functions 
for the manipulation of CUI variables. 
[0017] The registration of CUI variables is done automatically 
when the application uses special constructors. For ma- 
nipulation of CUI variables inside of the application 
(set/get/compare value(s)) we implemented special meth- 
ods and operators, which override analogous regular 
methods and operators used in C++ or Java. CUI DLL pro- 
vides these special constructors, methods and operators. 
For manipulation of CUI variables outside of the applica- 
tion (registration, visualization in GUI, modification of 
their values according the user's input) we use the set of 
methods, procedures and functions implemented in CUI 
Viewer. 

[0018] CUI Description is a text file used to define the application 
user interface as a set of descriptions of widgets with their 
respective sizes, locations, graphical and other attributes. 

[0019] | n our design CUI Viewer plays the role of a middleman 
between the application and its CUI, defined by GUI De- 
scription. It is used for the registration of CUI parameters, 
visualization and modification of the GUI(s) of one or sev- 
eral applications. 

[0020] During its start an application sends a CUI Description file 



name and its own name to CUI Viewer. If CUI Viewer is 
able to read CUI Description file, it converts its contents 
into a tree of widgets and displays them according their 
attributes. During its work the application generates each 
CUI variable by using a corresponding constructor from 
CUI DLL, and the constructor registers each variable with 
CUI Viewer. At this moment CUI Viewer attempts to create 
a link between each registered CUI variable and its corre- 
sponding widget. If it is successful, then the widget is up- 
dated. If not, CUI Viewer creates some default widget for 
the new CUI variable. This new widget is added to the 
chain of widgets and is displayed by CUI Viewer in the 
window "New Parameters". 
[00 21 ] Notes: 

[0022] i An application never needs the information about the 
current contents or even existence of CUI Description. 
Generally speaking, it should work fine even when this file 
does not exist. 

[0023] 2. If the CUI Description file name provided by application 
is empty, CUI Viewer uses the application name to con- 
struct some default file name for CUI Description. 

[0024] 3 All CUI variables of current application are linked to 
new widgets, if CUI Viewer cannot find or open CUI De- 



scription file. 

[0025] 4, The storage of all CUI variables is managed by CUI 

Memory Manager. In some implementations CUI Viewer 
uses Application Shared Memory Buffer (ASMB). This de- 
sign not only simplifies the treatment of CUI variables but 
make possible the use of common variables in different 
applications working with the same CUI Viewer. 

[0026] 5, CUI Viewer and CUI DLL do not depend on the purpose 
of the application. They could be written and/or compiled 
for any platform once or even become part of operating 
systems or Internet browsers. 

[0027] 5_ All functions and procedures of CUI Viewer could be 
easily moved into CUI DLL. In this case this DLL could be 
statically linked to any application program. 

[0028] 7, it is very natural to use XML representation for CUI De- 
scription. However, since today XML is not very well sup- 
ported on different platforms our program is capable of 
reading and writing CUI Description either in XML or in 
our own internal text format. 

[0029] when the application is running the user can: 

[0030] a ) change different attributes of any widget: size, position, 
color etc. 

[0031] b) move widgets from one window to another; 



[0032] c ) convert any widget to another widget type; 

[0033] d) construct new and delete old links between GUI vari- 
ables and widgets; 

[0034] e ) add new widgets not connected to any application vari- 
ables and which can be used, for example, as a new win- 
dow or button to open some window; 

[0035] f) save current widget configuration as the GUI description 
in the text file that will be used by CUI Viewer during the 
next start of application. 

[0036] An example of a simple "Hello World" application is pre- 
sented by FIG. 1. 

[0037] For simplicity we suppose that file "HelloWorld.cpp" was 
compiled into "HelloWorld.exe" application. When this ap- 
plication starts running it calls constructor CUIChar(char *) 
to initiate the editable CUI variable Param of character 
string type. This variable is registered with CUI Viewer un- 
der the name "Test". Then the application sets the value of 
Param to string "Hello World". Finally, it waits in a loop 
until the value of Param becomes equal to the string 
"End". 

[0038] when this application starts for the first time (or if Appli- 
cation CUI Description file is not presented), CUI Viewer 
displays an editable variable named "Test" having the 



value "Hello World" by using the default widget designed 
for CUI Variables of character string type in the default 
window "New Parameters" (FIG. 2) and waits for user in- 
put. 

[0039] In this example the text "Hello World" is the value of CUI 
variable Param, which can be modified by user. When user 
inputs the text "End", the application will end its execu- 
tion. 

[0040] without exiting the application by using the capability 
provided by CUI Viewer, user can create a new window 
"Test Window" and convert the initial default widget previ- 
ously used for representation of CUI variable Param into a 
pull-down menu "Hello World" with two options: "Hello 
World" and "End" (FIG. 3). 

[0041] if this is done, the application stops its execution when 
the user selects the option "End" from the described pull- 
down menu. At this point CUI Viewer asks user to save 
new (converted) CUI Description. If the user agrees, CUI 
Viewer saves the new CUI Description for "HelloWorld.exe" 
in a text file with the name "HelloWord.CUl". CUI Viewer 
will use this file during the next launch of application "Hel- 
loWorld.exe" and this application GUI will show up as in 
FIG. 3. 



[0042] | n this example (FIG. 1) the property of CUI variable Param 
being editable is defined by the default value of one pa- 
rameter of constructor CUIChar(char *). The full set of pa- 
rameters for this constructor will be described later. 

[0043] Any CUI variable has two mandatory and unchangeable 

attributes "Name" and "Type", which distinguish it among 
the other CUI variables. In other words, CUI Viewer will 
create only one CUI variable for any given pair (Name, 
Type) and it will provide read/write access to this variable 
attributes for any application working under its supervi- 
sion. 

[0044] The attribute "Name" is a non-empty character string. 

Currently CUI DLL and CUI Viewer support the following 

CUI variable types: 
[° 045 ] a) CUIint for integers; 

[0046] b) CUIdouble for doubles; 

[0047] c ) cuichar for character strings; 

[0048] d) cUlintArray for arrays of integers; 

[0049] e ) cUldoubleArray for arrays of doubles; 

[0050] f) cUlcharArray for arrays of strings. 

[0051] CUI DLL provides read/write access to the value of any CUI 



Variable and other manipulation of CUI variables by using 
a set of overloaded operators defined separately for each 
type in the corresponding C++ class. 
[0052] t wo cui Variables defined by one or several applications 
with the same pair of attributes (Name, Type) always have 
references to the same address provided by CUI Memory- 
Manager. 

[0053] FIG. 4 shows an example of a program for the calculation 
of the sum of two numbers. 

[0054] This program defines two user editable CUI variables 
FirstNumber and SecondNumber and CUI variable 
SumOfTwoNumbers that can be modified only by the ap- 
plication program. These three variables have the names 
"A", "B" and "Sum of A and B", respectively. Notice that the 
application program does not define how and where these 
variables must be displayed. 

[0055] The application program sets the initial values and then 
stays in awhile loop and calculates the sum of these two 
numbers. The function CUISIeepO belongs to CUI DLL. It 
waits for the requested number (100) of milliseconds and 
then checks the status of the application. If application 
program state is "End Application", then CUISIeepO returns 
1. The CUI DLL function supporting "set value" function 



sends the message "Update Value" to the CUI Viewer only 
if the value of SumOfTwo Numbers was actually changed. 

[0056] when the application starts for the first time (or if file 
"SumOfTwoNumbers.Clir is not found) the user is pre- 
sented with Initial state of window "New Parameters" (FIG. 
5). User can modify the values of A and/or B and 

[0057] a ) CUI DLL will modify FirstNumber and/or SecondNum- 
ber; 

[0058] b) application will calculate the result A + B; 

[0059] c ) CUI Viewer will automatically display new result (FIG. 6) 

[0060] Let's describe the hypothetical sequence of user's actions 
modifying the GUI of this application when it is running. 

[0061] i. user can create new window "Sum of A and B" and 

move all widgets representing CUI variables into this win- 
dow (FIG. 7). 

[0062] 2. User can duplicate widgets representing the values of A 
and B (FIG. 8). 

[0063] 3, user can modify size and display style of the first wid- 
get representing A (B) and convert the second widget rep- 
resenting the same CUI variable into a slider (FIG. 9). 

[0064] 4, user can test the newly created interface by using slid- 
ers to modify the values of editable CUI variables A and/or 



B. 

[0065] 5, Finally, user can save the new CUI Description of the 
newly constructed interface and see it in the next launch 
of the application. 

[0066] Let's describe the data structures we are using to make 
CUI work. For this we will rather use "functional" descrip- 
tions of class members and not their formal names. 

[0067] v\/e are using three main base classes: CUIParamEntry, 
CUIParam and CUIObject. The first two of these contain 
information about CUI variables and methods for their 
manipulation; the last contains information about widgets. 

[0068] FIG. 10 shows the information stored by any instance of 
CUIParamEntry class. 

[0069] "Actual Value" (FIG. 10 line 3) can be changed either by CUI 
Application. Note that CUI Viewer can change "Actual 
Value" only if an application defined the associated CUI 
variable as CULEDITABLE and the user changes the corre- 
sponding field of the widget linked with the variable. Each 
CUI variable is an instance of a type specific class (CUIint, 
CUIdouble or CUIchar) derived from class CUIParam. Any 
instance of CUIParam contains only one member "Index of 
Parameter", which is the index of an instance of class 
CUIParamEntry previously allocated by CUI MemoryMan- 



ager. 

[0070] The constructor of class CUIParam uses one of its input 
parameters, Name, passed to it by the constructor of one 
of the derived classes (CUIint, CUIdouble or CUIchar), and 
corresponding Type and attempts to find an instance of 
CUIParamEntry with the same Name and Type in the mem- 
ory allocated by CUI Memory Manager. In case of a match 
it makes Index of Parameter equal to the index of found 
instance. Otherwise the new instance of CUIParamEntry 
with given (Name, Type) is constructed and Index of Pa- 
rameter stores its index. 

[0071] All constructors of CUIint, CUIdouble, and CUIchar classes 
pass additional parameters to the constructor of CUIParam 
class, which define the initial value of constructed CUI 
variable and some attributes. 

[0072] FIG. 11 shows the information stored by any instance of 
CUIObject class. 

[0073] v\/e use the following three classes derived from CUIObject 
to represent different types of widgets: CUIEdit, CUIButton 
and CUIPicture. Each of these classes contains the full de- 
scription of graphical representation and location of cor- 
responding widget. 

[0074] Th e operation of any application working in CUI environ- 



ment is based on the message exchange between some 
functions provided by CUI DLL, CUI Viewer and the special 
thread automatically launched for the application. Among 
these messages the most important are the "SET_ VALUE" 
and " RU N _C ALLB AC K" messages. The first of these initial- 
izes the changes in the visualization of CUI Variables on 
the screen, and the other initializes some reactions of the 
application related to the user's input. 
[0075] Every time the value of any CUI Variable (CULVar) is 

changed, a special CUI DLL function sends a "SET_ VALUE" 
message with the address of corresponding instance of 
CUIParamEntry class to CUI Viewer. Upon receiving this 
message CUI Viewer performs the following sequence of 
operations: 

[0076] a) It checks the value of the field "CUIObject Address" (FIG. 
10 line 11). If it is NULL, CUI Viewer searches for an in- 
stance of CUIObject class with the same Name (FIG. 10 
line 1) and Type (FIG. 10 line 2) in the tree of CUIObjects. 
If no such instance is found, the CUI Viewer creates a new 
instance of CUIObject class with the default parameters 
for the visualization of CULVar and address of the 
CUIParamEntry instance stored in "Reference to parame- 
ter" field (FIG. 11 line 19). The address of the newly ere- 



ated or found instance is stored in the field "CUIObject 

Address" (FIG. 10 line 11). 
[0077] b) It displays the new value of CULVar by using the virtual 

function Draw() for the instance of CUIObject linked with 

CULVar. 
[0078] N otes: 

[0079] i) when the search of CUIObject instance for current 

Name (FIG. 10 line 1) and Type (FIG. 10 line 2) is unsuc- 
cessful, the default parameters for visualization of 
CULVar by newly constructed CUIObject instance make 
possible the visualization of CULVar in the special window 
"New Parameters". 

[0080] 2) Any CUI Variable can have more than one representa- 
tion in GUI described by different CUIObject instances. All 
these instances comprise a doubly linked list (FIG. 11 lines 
34, 35) used by CUI Viewer when it processes the 
"SET_ VALUE" message. 

[0081] when the application calls any CUI function for the first 
time, CUI Viewer starts a special thread CUIAppThread 
used for waiting and processing of "RUN_CALLBACK" mes- 
sages. CUI Viewer sends such a message every time the 
value of CUI Variable CULVar is modified by user's inter- 
action with the GUI (or by another program) and the field 



"Callback address" (FIG. 10 line 5) of the CUIParamEntry 
instance associated with CULVar contains a non-NULL ad- 
dress of some callback routine of the application. 

[0082] | n the special "Edit" mode CUI Viewer allows the user to 
convert any widget into a widget of another type or 
change its attributes. "Edit" mode can be selected by 
pressing "Ctrl-E" ("Ctrl"+ "E" buttons simultaneously). 
When this is done the user should: 

[0083] a ) select the widget he wants to convert or modify by 
making a right mouse button click inside this widget; 

[0084] b) open the CUI Viewer pull down menu "Convert To..." 
and select the new widget type by pressing a button in- 
side desired widget type image. 

[0085] FIG. 12 shows the full list of CUI Viewer commands cur- 
rently supported in "Edit" mode. 

[0086] Notes: 

[0087] i) CUI Viewer itself is written as CUI application; therefore 
the user can add or modify the list of widgets in the "Con- 
vert To" menu. 

[0088] 2) CUI Viewer does not send any messages regarding the 
widget conversion to the application program, and the ap- 
plication continues its execution as usual. 

[0089] The user can record input information to file and then 



play this file by using CUI Viewer commands "Record" and 
"Play" (FIG. 12, lines 11 and 12). In "Record" mode CUI 
Viewer stores only information about CUI Variables 
changes. Each record contains following information: 
[0090] 1. Variable Name; 

[0091] 2. Variable Value; 

[0092] 3 Time when user changed Variable Value. 

[0093] The CUI Viewer "Play" command can be used even if the 
positions, sizes or types of any Widgets have been 
changed (claim 12). During Play CUI Viewer reads these 
records and simulates execution (moves mouse pointer to 
widget with the same Variable Name as in the record and 
changes Variable Value) according to recorded time. 

[0094] |f there is more than one widget with the same name, CUI 
Viewer calculates the "Time To Set Value" for each such 
widget. The "Time To Set Value" is calculated as the time 
needed to move the mouse cursor at "Mouse Simulation 
Speed" to the closest point of the widget plus the time for 
"Value Simulation Change". If the widget is not visible (for 
example, it is located in a closed window), the "Time To 
Set Value" is equal to the time needed to move mouse 
cursor and make this widget visible plus time needed for 



"Value Simulation Change". The widget with minimum 
"Time To Set Value" is used. 
[0095] | n the current implementation the default value of "Mouse 
Simulation Speed" is equal to 5 centimeter per second, 
"Time To Set Value" is equal to number of the Mouse/ 
Keyboard button clicks multiplied by 0.3 seconds per 
click. 

[0096] Some methods of current invention are based on or inten- 
sively use the external and internal representation of dif- 
ferent CUI data in XML format. All these XML-methods can 
be described as "static" or "dynamic" according to their 
usage of transformation of XML representations involved 
in the corresponding processes. 

[0097] The static XML-methods use XML representation for stor- 
age and reading/writing procedures of CUI Description 
(claim 7) and widget library (claims 13, 17). They use no 
transformations of XML data. 

[0098] Every dynamic method uses some XSL transformation to 
convert the initial XML representation of some part of CUI 
data into its new XML representation. When the transfor- 
mation is complete the CUI Viewer uses this new XML rep- 
resentation for visualization of the same CUI data in some 
other (converted) way. This approach is implemented for 



automatic recalculation of coordinates and sizes of wid- 
gets (claim 14) and for automatic rule based conversion of 
attributes of replaced widgets (claim 15). 
[0099] N otes: 

[0100] i) The XSL files used for our dynamic XML-methods are 
the external text files that form the special Transforma- 
tion Library. Any of these files can be modified or replaced 
by a new XSL files, which can provide new rules for some 
transformation (claim 16). 

[0101] 2) The use of a special naming convention for XSL files of 
Transformation Library makes it possible to support pre- 
defined lists for widget conversion (claims 8, 9) 

[0102] 3) The use of this naming convention makes the Transfor- 
mation Library expandable. There is a possibility not only 
to replace some or all XSL files, but also to add new XSL 
files providing conversion rules for a new customer's wid- 
gets. 

[0103] 4) The names of CUI Variables can be used in XSL trans- 
formation files names. 

[0104] 5) while processing an XSL file, a special function from 
CUI DLL has access to the current value of any CUI Vari- 
able registered with CUI Viewer. This makes it possible to 
use not only the names of CUI Variables but also their val- 



ues during XSL transformation. 

[0105] 



